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Abstract 

We study the design of mechanisms satisfying two desiderata — incentive compatibility 
and privacy. The first, requires that each agent should be incentivized to report her private 
information truthfully. The second, privacy, requires the mechanism not reveal 'much' about 
any agent's type to other agents. We propose a notion of privacy we call Joint Differential 
Privacy. It is a variant of Differential Privacy, a robust notion of privacy used in the Theoretical 
Computer Science literature. We show by construction that such mechanisms, i.e. ones which 
are both incentive compatible and jointly differentially private exist when the game is 'large', 
i.e. there are a large number of players, and any player's action affects any other's payoff by 
at most a small amount. Our mechanism adds carefully selected noise to no-regret algorithms 
similar to those studied in Foster- Vohra [FV97] and Hart-Mas-Colell [HMC00]. It therefore 
implements an approximate correlated equilibrium of the full information game induced by 
players' reports. 
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1 Introduction 



In several situations of interest, information relevant to computing a socially desirable outcome is 
dispersed among the agents in society. The fields of mechanism and market design study both the 
feasibility of, and how to provide incentives to implement a desired outcome. By designing an 
appropriate 'mechanism', agents with relevant private information are incentivized to reveal it, and 
thus an appropriate outcome can be selected and implemented. We revisit this with an additional 
desideratum motivated by privacy concerns — that each agent's information not be revealed to any 
other agent, either directly or by the portion of the outcome that is revealed to any agent. 

It is easiest to understand our question via a (slightly futuristic) motivating example. Imagine a 
city in which (say) Google Navigation has become the dominant navigation service, with universal 
adoption. Every morning, each person in this city enters their starting point and destination into 
their Google device. Each person then receives a set of directions, and chooses his/her route 
according to those directions. In this paper we consider the design of an mechanisms that given 
everyone's reports of start and end points provides each agent a suggested route, satisfying two 
desiderata: 

1. Incentive Compatibility: This is the standard concern in the literature on mechanism 
design. Each agent should be incentivized to report his starting and end points 'truthfully', 
and then follow the driving directions provided. Both of the following deviations should be 
ruled out for each agent — (i) misreporting start and end points, and (ii) reporting start and 
end points, but following a different (shorter) path. 

2. Privacy: Players desire the privacy of their starting and end points. In other words, the 
mechanism should be such other player or players cannot infer 'much' about a given player's 
source or destination based on the directions they received. 

Intuitively, these two desiderata are hard to satisfy simultaneously. If there are a small number 
of players and a small number of routes, an agent may be able to infer others' source-destination 
pairs from the suggested route she receives. Conversely, routes that guarantee privacy (e.g. the 
recommended route given to a player is selected independently of others' reports) may be very 
different from 'equilibrium' routes. To circumvent these difficulties, we restrict attention to a 
specific class of environments satisfying two conditions: 1) Private values, i.e. an agent's private 
information only influences her own payoffs, 2) the game is 'large', i.e. there are a large number 
of agents and agents' payoffs are 'insensitive' to any single other agent's choice of action. Further, 
we weaken the equilibrium concept. Our mechanism's suggestions to each player constitute an 
approximate equilibrium of the full information game induced by their reports. 
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Our mechanisms can be described via simple algorithms, and are based on a combination of 
two ideas from the literature. The first of these ingredients is the use of 'no-regret methods' to 
compute equilibria (see Foster- Vohra [FV97] and Hart-Mas-Colell [HMCOO]). Roughly speaking, 
in these methods, in each period, each player gets feedback on his losses, i.e. counterfactual losses 
(or gains) in payoff if the player had played a different action than the one he did. He then updates 
his action to minimize his 'regret'. Viewed as a centralized algorithm simulating this process for 
each player, this algorithm computes an approximate equilibrium of the game. 1 

The second ingredient is noise. At each stage of this simulation, our algorithm uses 'noisy' 
losses, i.e. the actual loss plus appropriately chosen random noise, instead of the true loss. By 
carefully choosing this noise, we are able to guarantee that 1) agents have incentives to play 'truth- 
fully' in this game, 2) the outcome maintains the privacy of each agent's private information, and 3) 
for each profile of reported types, the algorithm implements an outcome which is an approximate 
equilibrium of the induced full information game. 

1 . 1 Overview of Model and Results 

More formally, we consider a setting with non-transferable utility. Each agent has a finite set of 
actions, and private information about his own payoff type. Our setting is therefore a 'private 
values' setting- agents' payoffs depend on the actions taken by everyone, as well as their private 
type; other agents' types do not directly influence their payoffs. 

A centralized planner simultaneously receives reports of type from each agent and proposes an 
action to each. We study the design of mechanisms that: 

1. Propose an approximate equilibrium of the full information game given the reports (akin 
to, for example, the literature on two-sided matchings where mechanisms implement a full 
information concept, stability, even though the underlying setting is one of incomplete infor- 
mation). Our solution concept here is e-correlated equilibrium. 2 

2. Make it an approximately-dominant strategy for agents to report truthfully. 

3. Preserve the privacy of each agent's private information. Our first contribution is a definition 
of what it means to compute an equilibrium privately. This is new to the literature, and is 

'There are multiple notions of regret. Depending on the type of regret minimized, the algorithm converges to either 
approximate correlated or approximate coarse correlated equilibrium of the given game. 

2 For certain classes of games, this can be extended to e-Nash equilibria. The main constraint is our proof technique. 
We need that the solution concept must be computable by an appropriate distributed algorithm, to which we can add 
carefully calibrated noise. In certain special cases, these conditions are satisfied for Nash equilibrium, but in this paper 
we restrict our attention to correlated equilibrium so as to maintain generality. 
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an extension of differential privacy which is a robust (i.e. demanding) criterion. Roughly 
speaking, it requires that no-one learn much about the type of any agent, even if he knows 
'everything else', i.e. everyone else's realized type and recommended actions. We give a 
definition of differential privacy adapted to our setting in Definition 5, see also discussion 
and related work in Section 1.2. 

It is easy to see that the goal of computing an approximate equilibrium while preserving the 
privacy of the player's utility functions is hopeless in a 2-player game (or more generally a small 
number of players). Therefore we consider 'large' n-player games. We define these formally 
in Section 2, but roughly speaking, these are n-player games in which for all players i ^ j, i's 
choice of action can affect j's payoff by at most an additive ±7. We call 7 the sensitivity of the 
game. In what follows, we discuss our results for games where 7 is 0(l/n), e.g. anonymous 
matching games or more generally games where a player's payoff depends on his own action and 
the distribution of others' actions. 

We consider two equilibrium concepts: coarse correlated equilibrium (CCE), and correlated 
equilibrium (CE). We also give a computationally efficient algorithm for privately computing a- 
approximate versions of both solution concepts where a is O (poly (A;) / ' yfn) . Therefore for games 
with a fixed number of actions, the approximation is 0{l/y/n). 

For games with a 'large' number of actions the above algorithm is not useful, due to the poly (A;) 
term in the numerator. For example, in the routing game discussed earlier, the number of actions 
available to a player is the number of paths, which can be exponentially large relative to the size 
of the graph. For such settings with large numbers of actions, we show that positive results are 
still possible as long as the number of possible types for each player is bounded. Formally, we 
show that it is possible to 'privately' compute an a-approximate equilibrium in a large A;-action n- 
player game, with U feasible utility functions for each player, where a is 0(log k log 3 / 2 \U\/y/n). 
However, the algorithms for this are computationally inefficient. 

How tight are our bounds? In other words while our mechanism identifies a certain tradeoff be- 
tween privacy and approximation, perhaps one could do better? We answer in the negative- we also 
show a matching lower bound: we give a family of n-player 2-action large games in which it is not 
possible to privately compute an a-approximate CCE (and therefore an a-approximate CE or an 
ct-approximate Nash equilibrium) for a <C 1/ y/n, showing that even our efficient algorithm gives 
nearly the best possible approximation guarantees in the case that k is small (i.e. a fixed number in- 
dependent of n). Our inefficient upper bound of course remains tight up to a factor of log k log 3 / 2 U 
for arbitrary A>action games with U feasible utility functions. Whether there is an efficient algo- 
rithm for privately computing ^-approximate equilibria to error a = 0(polylog(A;, U) / y/n) is left 
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as an open question. 

Finally, what do these results mean in terms of incentive properties? It has been observed previ- 
ously that differential privacy implies approximate strategy proofness [MT07]. Roughly speaking, 
since a player's report cannot reveal 'much' to anyone else, it must be the case that the distribution 
over suggested actions to everyone else cannot change by 'much' as a function of any players' 
report. Therefore, e-jointly differentially private mechanism is also e-strategy proof: it is an e- 
dominant strategy for any player to report her true type. Since the actions proposed jointly consti- 
tute a a-approximate correlated equilibrium of the full information game given everyone's reports; 
it is a e + a-approximate Nash equilibrium for everyone to follow the strategy "truthfully report 
type, then follow the recommended action" 3 . We show that it is possible for both e and a to asymp- 
tote to 0, i.e. for any arbitrarily small degree of privacy e and approximateness of equilibrium a, 
there exists n large enough such that our mechanism that guarantees e privacy and a -approximate 
equilibrium. Therefore, as the size of the game grows large, truthfully reporting type and following 
the suggested action approaches an exact Nash equilibrium of the full information game. 

1.2 Related Work and Discussion 

Market and Mechanism Design Our work is closely related to the large body of literature 
on mechanism/market design in 'large games'. This literature looks to exploit the large number of 
agents to provide mechanisms which have good incentive properties, even when the small market 
versions do not. It stretches back to Roberts & Postlewaite [RP76] who showed that market (Wal- 
rasian) equilibria are approximately strategy proof in large economies. More recently Immorlica 
and Mahdian [IM05] , Kojima and Pathak [KP09], Kojima, Pathak and Roth [KPR10] have shown 
that various two-sided matching mechanisms are approximately strategy proof in large markets. 
There are similar results in the literature for one-sided matching markets, market economies, and 
double auctions. Azevedo and Budish [AB11] in a recent paper provide conditions for a mecha- 
nism to be 'strategy proof in the large', i.e. approximately strategyproof as the game grows large. 

By comparison with these works, which study settings where the mechanism designer/princi- 
pal can enforce outcomes (or take actions on behalf of participants), we study settings where the 
mechanism only suggests an action to participants. This leads to slightly weaker incentive proper- 
ties (due to the possibility of 'double-deviations'). Indeed, if our mechanism could act on behalf 
of participants, it would be (e + a) -approximately strategy proof when a a-approximate correlated 

3 It is always an e-approximate Dominant strategy to truthfully report type. It is an e + a Nash equilibrium to follow 
both parts of the two-part strategy, of truthfully reporting, and then following the resulting suggested equilibrium 
action. 
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equilibrium is computed while satisfying e-differential privacy. 4 

On a related subject, there is literature suggesting that even if the mechanism can enforce 
outcomes rather than only suggest an action, other considerations may require the mechanism 
to select a 'equilibrium' outcome of the underlying game rather than an 'optimal' outcome. An 
influential body of work, starting with Roth and Xing [RX94] argues that in two-sided matching 
markets, centralized mechanisms that implement a stable outcome (a full information solution 
concept) are more resistant to unraveling, i.e. members of the market pre-empting the mechanism 
by contracting in advance. 

Large Games Our results hold under two sufficient (and almost necessary) conditions: that 
the number of players be 'large', and the game be insensitive to 0(l/y/n), i.e. a player's action 
affects the payoff of all others by a small amount. These are closely related to the literature on 
large games, see e.g. Al-Najjar and Smorodinsky [ANSOO] or Kalai [Kal04]. There has been 
recent work studying large games using tools from theoretical computer science (but in this case, 
studying robustness of equilibrium concepts)- see Gradwohl and Reingold [GR08, GR10]. 

Differential Privacy Differential privacy is a recently proposed formalization of privacy. 
It was first defined by Dwork, McSherry, Nissim, and Smith [DMNS06], and has since become 
the standard privacy "solution concept" in the theoretical computer science literature. It is a quan- 
tification of the worst-case harm that can befall an individual as a result of his decision to allow 
his data to be used in some computation, as compared to if he did not provide his data. Roughly 
speaking, an algorithm is e-differentially private if adding/removing/changing the data of a single 
individual to a dataset can change the probability of any outcome of the algorithm's computation 
by at most a (1 + e) factor. Note that this is a worst case notion. It requires this bound even if the 
adversary knows the rest of the dataset. 5 

There is by now a very large literature on differential privacy, which we will not attempt to 
survey. Instead, we mention here only the most relevant work. Interested readers can browse the 
lecture notes at [Rotl 1] for a more thorough introduction to the field. 

The most well studied problem in differential privacy is that of accurately answering numeric- 
valued queries on a data set. A basic result is that any single query that has sensitivity at most 1 
(i.e. the addition or removal of a single individual from the data set can change the value of the 

4 ln fact, if the participants did not have the option of acting independently of the mechanism (i.e. still playing the 
game, but selecting an action without consulting the mechanism), then our mechanisms would be e-strategyproof. 

5 An example may be useful. Suppose each individual knows his age, and the algorithm computes the average of 
all the items in the data set. This is not differentially private- someone who knows the output of the algorithm can 
deduce the age of any one individual (if he knows all others' ages). Differential privacy thus requires that the mean be 
reported with appropriately selected random noise. 
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query by at most 1) can be answered in a computationally efficient manner while preserving e- 
differential privacy, and introducing error only 0(1/ e) [DMNS06]. Another fundamental result in 
differential privacy is that it composes gracefully: Any algorithm composed of T subroutines, each 
of which are 0(e/yT) -differentially private is itself e-differentially private [DKM+06, DRV10]. 
Combined with the previous result, this gives an efficient algorithm for privately answering any T 
low sensitivity queries with error that grows only with O(VT), a result which we make use of. 

Another line of work has shown that it is possible to privately answer queries much more ac- 
curately using computationally inefficient algorithms [BLR08, DRV10, RR10, HR10, GHRU11, 
GRU12]. Combining the results of [RR10, HR10] yields an algorithm which can privately answer 
arbitrary low sensitivity queries, interactively as they arrive, with error that scales only logarith- 
mically in the number of queries. We make use of this when we consider games with large action 
spaces but small type spaces. 

There is also a line of work proving information theoretic lower bounds on the accu- 
racy to which low sensitivity queries can be answered while preserving differential privacy 
[DN03, DMT07, DY08, Del2]. Our lower bounds for privately computing equilibria work by 
reducing the problem to privately answering queries: we design a game whose only equilibria 
encode answers to large numbers of queries about a database. 

Finally, related to this paper, there is a recent literature on connections between differential pri- 
vacy and game theory. McSherry and Talwar [MT07] were the first to observe that a differentially 
private algorithm is also approximately truthful, and to use this fact to design approximately truth- 
ful mechanisms for an unlimited supply auction setting. This line of work was extended by Nis- 
sim, Smorodinsky, and Tennenholtz [NST12] to give mechanisms in several special cases which 
are exactly truthful (although no longer privacy preserving) by combining private mechanisms 
with non-private mechanisms which explicitly punish non-truthful reporting. Huang and Kannan 
[HK12] showed that the mechanism used by Mcsherry and Talwar (the "exponential mechanism") 
is in fact maximal in distributional range, and so can be made exactly truthful with the addition 
of payments. We remark that the immediate connection between privacy and approximate incen- 
tive compatibility leveraged by these works only holds in settings in which the mechanism has the 
power to enforce its outcome or otherwise compel actions. The novelty in our work relative to 
this line is that our mechanisms implement approximate equilibria of the full information game. 
Therefore, truthful reporting and subsequently following the suggested equilibria actions remain 
approximate best responses even if the players have the ability to act in the game, independently of 
the mechanism. Our results are therefore very general: we are able to implement our mechanisms 
in arbitrary large games, without requiring that the mechanism designer claim authority to enforce 
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actions. 

Another interesting line of work considers the problem of designing truthful mechanisms 
for agents who explicitly experience a cost for privacy loss as part of their utility function 
[Xiall, NOS12, CCK + 11]. The main challenge in this line of work is to formulate a reasonable 
model for how agents experience cost as a function of privacy. We remark that the approaches taken 
in [NOS12] and [CCK + 11] can also be adapted to work in our setting, for agents who explicitly 
value privacy. Gradwohl [Gral2] studies the problem of implementation for various assumptions 
about players' preference for privacy and permissible game forms. A related line of work which 
also takes into account agent values for privacy [GR11, FL12, RSI 2, LR12] considers the prob- 
lem of designing markets by which analysts can procure private data from agents who explicitly 
experience costs for privacy loss. See Roth [Rotl2] for a survey. 

1.3 Organization of this Paper 

Section 2 outlines the model and gives a formal definition of differential privacy for our setting. It 
also defines and lays out some known results about our main workhorse for this paper, no-regret 
algorithms. Section 3 shows that no-regret algorithms are 'tolerant' to certain types of noise, i.e. 
they still converge efficiently. Section 4 then argues that with appropriately chosen noise, the 
output of the no-regret algorithm will be differentially private, but still converge to an approximate 
equilibrium. It formally lays out the tradeoff between the various parameters we define, i.e. the 
sensitivity of the game, the degree of privacy required, how approximate the equilibrium is and the 
number of players. It then argues the incentive properties of this mechanism. It then shows that 
our bounds are tight, i.e. there cannot exist a mechanism that maintains the same level of privacy 
but implements a less approximate (i.e. more exact) equilibrium. 

2 Model & Preliminaries 

There is a set of n players, {1,2,..., n}, the generic player is indexed i. Player i can take actions 
in a set A, \A\ = k. 6 We denote a generic action by j and a generic action for player i by eij. A 
tuple of actions, one for each player, will be denoted a = (ai, a 2 , . . . a„) G A n . 7 

Player i's payoff function will be denoted Ui : A n — > 3ft. We will restrict attention to 'insensi- 
tive' games. Roughly speaking a game is 7-sensitive if a player's choice of action affects any other 

6 It is trivial to extend our results to the case where agents have different sets of actions, k will then be an upperbound 
on the number of actions across agents. 

7 In general, subscripts will refer indices i.e. players and periods, while superscripts will refer to components of 
vectors. 
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player's payoff by at most 7. Formally: 

Definition 1 (7-Sensitive). A game is said to be ^-sensitive if for any two distinct players i, i', 
any two actions a^, a[for player % and any tuple of actions a_ifor everyone else: 

\ui>(ai,a-i) - w;/(a-,a_i)| < 7. (1) 

We should note that the interpretation of several of our results will depend on 7 being 0(l/n). 
This is a satisfied by, for e.g. anonymous matching games, and is standard in the large games 
literature. Note, however, that in general our results will be non-trivial so long as 7 = o(l/ \/n). 

Denote a distribution over A n by 7r, the marginal distribution over the actions of player i by 7T;, 
and the marginal distribution over the (joint tuple of) actions of every player but player i by 7r_j. 
We now present (approximate versions of) two standard solution concepts — correlated and coarse 
correlated equilibrium. 

Definition 2 (Approximate Coarse Correlated Equilibrium). Let (ui,u 2 , ■ ■ - u n ) be a tuple of 
utility functions, one for each player. Let 71 be a distribution over tuples of actions A n . We say that 
7r is an a-approximate coarse correlated equilibrium of the game defined by (ui,u 2 , . . . u n ) if for 
every player i, and any a[ G A: 

E[itj(a)] > E [uj(a-, a_j)] - a 

TT 7T_i 

Definition 3 (Approximate Correlated Equilibrium). Let (u\, u 2 , ■ ■ ■ u n ) be a tuple of utility func- 
tions, one for each player. Let n be a distribution over tuples of actions A n . We say that n is an 
a-approximate correlated equilibrium of the game defined by (ui, u 2 , ■ ■ ■ u n ) if for every player 
i G [N], and any function f: A — >■ A, 

E K(a)] > E [ui(f(ai), a_j)] - a 

TV TV 

Let U be the set of all possible utility functions for the players, 8 with a generic profile of 
utilities u = (ui, u 2 , ■ ■ ■ u n ) ElA n . A mechanism is a function from a profile of utility functions to 
a probability distribution over 1Z n , i.e. M:U n ^ A1Z n . Here 1Z is an appropriately defined range 
space. 

First we recall the definition of (standard) differential privacy, both to provide a basis for our 
new definition, and since it will be a technical building block in our algorithms. Roughly speaking, 

8 It is trivial to extend our results to the case where agents have different sets of possible utility functions, Ui. U 
will then be U™=i 
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a mechanism is differentially private if for every u and every i, knowledge of the output Ai(u) as 
well as u_i does not reveal 'much' about u^. 

Definition 4 ((Standard) Differential Privacy). A mechanism M. satisfies (e, 5) -differential pri- 
vacy if for any player i, any two possibility utility functions for player i, Ui and u[, and any tuple 
of utilities for every else w_ j and any S C TZ n , 

P [(M( Ul ; u _0) e5]< e £ P \{M(u'a eS] + 5. 

M M 

We would like something slightly different for our setting. As we suggested earlier M. com- 
putes an equilibrium of the game specified by u. Each player reports his utility function to the 
mechanism. Given a profile of reports u, a n-dimensional vector in range space 1Z is drawn ac- 
cording to the distribution Ai(u). Player i is then given the i'th component of the drawn vector, 
which roughly corresponds to his 'recommended' strategy. 

Given this, we propose a relaxation of the above definition, motivated by the idea the action 
recommended to a player is only observed by her. Roughly speaking, a mechanism is jointly 
differentially private if, for each player i, knowledge of the other n — 1 recommendations (and 
submitted utility functions) does not reveal 'much' about player i's report. Note that this relaxation 
is necessary in our setting, since knowledge of player i's recommended action can reveal a lot of 
information about his utility function. It is still very strong- the privacy guarantee remains even if 
everyone else colludes against a given player i, so long as i does not himself make the component 
reported to him public. 

Definition 5 (Joint Differential Privacy). A mechanism M. satisfies (e, 5)-joint differential pri- 
vacy if for any player i, any two possible utility functions for player i, m and u[, any tuple of 
utilities for everyone else U-i and S C TV 1 ^ 1 , 

P [{M{ui\ eS]< e e P [(.M«; «_*))_, e S] + 5. 

An important result we will use is that differentially private mechanisms 'compose' nicely, i.e. 
that putting together multiple differentially private mechanisms even in an adaptive (rather than 
fixed) way, still results in a differentially private mechanism. 

Definition 6 (Adaptive Composition [DRV10]). Let u e U be a tuple and A: U -> TZ T be any 
algorithm. We say A is a T-fold adaptive composition of (e, 5) -differentially private mechanisms if 
there exists another algorithm B such that A(u) can be written as follows — for each t = 1, . . . , T: 

1. B{M 1 ,r x ,...,Mt-ur t „ 1 ) = M t . 
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2. M t is an (e, 5) -differentially private mechanism. 

3. r t is a draw according to Ai t (u), r t G 1Z. 

4. The output A(u) = (r l5 r 2 , . . . r T ). 

Note here that the choice of the t th mechanism is not fixed a priori and can depend on the output 
up to t — 1. Hence the 'adaptive' nomenclature. 

Theorem 1 (Adaptive Composition [DRV 10]). Let A: U — > 1Z T be a T-fold adaptive composi- 
tion of (e, 5) -differentially private mechanisms. Then A satisfies (e', T5 + 5')- differential privacy 
for 

e' = £y/2T]n(l/6') + Te(e e - 1). 

In particular, for any e < 1, if A is a T-fold adaptive composition of [ej A/8Tln(l/5), 0)- 
differentially privacy mechanisms, then A satisfies (e, 5) -differential privacy. 

Finally, differentially private mechanisms often involve adding Laplacian random noise. 9 We 
will denote a (mean 0) and standard deviation a Laplacian random variable by Lap(cr). The fol- 
lowing well known theorem shows that adding Laplacian noise to a insensitive function makes it 
differentially private. It follows easily from Definition 4 and the distribution of Laplacian random 
variables. 

Theorem 2 (Privacy of Laplacian Noise). Let Q : U — >■ R be any ^-sensitive function. Define the 
mechanism M.{x\) = Q(u) + Lap(er). If a = 7/e, then M. is (e, 0)- differentially private. 

We state a known concentration inequality for Laplacian random variables that will be useful. 

Theorem 3 ([GRU12]). Suppose {Yi}J =1 are i.i.d. Lap(cr) random variables, and scalars qi G 
[0, 1]. Define Y :— k £\ q^Yi. Then for any a < a, 

Pr[y > a] < exp 

2.1 No-Regret Algorithms: Definitions and Basic Properties 

Here we recall some of the basics about no-regret learning. See [Nis07] for a text-book exposition. 

Let {1, 2, ... , k} be a finite set of k actions. Let L = . . . , l T ) G [0, l] Txfc be a loss matrix 
consisting of T vectors of losses for each of the k actions. Let IT = |tt G [0, l] k \ Y^=x 71_J = l} 

9 A mean Laplacian distribution is the distribution of the difference of two i.i.d. exponential random variables. 
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be the set of distributions over the k actions and let ttjj be the uniform distribution. An online 
learning algorithm A: II x [0, l] k — > II takes a distribution over k actions and a vector of k 
losses, and produces a new distribution over the k actions. We use A t (L) to denote the distribution 
produced by running A sequentially t — 1 times using the loss vectors li , . . . , Z t _i, and then running 
A on the resulting distribution and the loss vector l t . That is: 

A Q {L) = 7T U: 
A t (L)=A(At-i(L),lt). 

We use A(L) = (Aq(L), A\(L), . . . , At(L)) when T is clear from context. 

Let 7r , . . . , n T G II be a sequence of T distributions and let L be a T-row loss matrix. We 
define the quantities: 



A(7r,o=xyp, 

i t 

A(7T , . . . , 7T T , L) = - ^ A(7T t , Z t ), 

t=l 

A(^(L'), L) = \(ML'),ML'), ML'), L). 

Note that the notation retains the flexibility to run the algorithm A on one loss matrix, but measure 
the loss A incurs on a different loss matrix. This flexibility will be useful later. 

Let J 7 be a family of functions / : {1,2, ... ,k} — > {1,2,..., k}. For a function / and a 
distribution n, we define the distribution fon to be 



j'-J(j')=j 



7l J 



The distribution fon corresponds to the distribution on actions obtained by first choosing an action 
according to it, then applying the function /. 
Now we define the following quantities: 

A(7Ti, ...,7T T ,L,f) = A(/07Ti, /07T 2 , . . . , /o7T T , L), 

p(A,L,f) = \(A,L)-X(A,L,f), 
p(A,L,F) = max p(A,L,f). 

As a mnemonic, we offer the following. A refers to expected loss, p refers to regret. Next, we 
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define the families Jfi xe d > J^s 



swap • 



•F\ Fixed 



swap 



{./•(./') ./• forall/|j e{l,2,... 
{/:{1,2,...,*}->{1,2,. ..,*}} 



k}} 



Looking ahead, we will need to be able to handle not just a priori fixed sequences of losses, 
but also adapted. To see why, note that for a game setting, a player's loss will depend on the 
distribution of actions played by everyone in that period, which will depend, in turn, on the losses 
everyone experienced in the previous period and how everyone's algorithms reacted to that. 

Definition 7 (Adapted Loss). A loss function C is said to be adapted to an algorithm A if in each 
period t, the experienced losses l t G [0, l] fc can be written as: 



We will make use of the following well-known results from the theory of no-regret algorithms, 
which show the existence of algorithms that guarantee low regret even against adapted losses (see 
e.g. [Nis07]). 

Theorem 4. There exists an algorithm *4.fi xe d such that for any adapted loss C, 



p(-4fixed, £, Jiixed) < y 21< ^* k '' . There also exists an algorithm ^4 swap such that p(A swap , C, J-'swap) < 



2.2 From No Regret to Equilibrium 

Let («!,..., u n ) be utility functions for each of n players. Let S = {(^i, . . . , tTi,t)}^ =1 be a col- 
lection of n sequences of distributions over k actions, one for each player. Let . . . , /j,t)}™ =1 
be a collection of n sequences of loss vectors / G [0, l] k formed by the action distribution. More 



where Si = {ir i}0 , . . . , ir i>T ) and L t = (l i>u l i>T ). 

Given the collection S, we define the correlated action distribution Us be the average distribu- 
tion of play. That is, lis is the distribution over A n defined by the following sampling procedure: 
Choose t uniformly at random from {1, 2, ... , T}, then, for each player i, choose a, randomly 
according to the distribution ir i;t , independently of the other players. 



l t = C(k, Ah), h,A(h), l t -i, A{l t -i)). 






14 



The following well known theorem (see, e.g. [Nis07]) relates p max to the equilibrium concepts 
(Definitions 2 and 3): 

Theorem 5. If the maximum regret with respect to J-fi xe d is small, i.e. p m ax{S, L, J-~f\ X ed) < ct> then 
the correlated action distribution lis is an a- approximate coarse correlated equilibrium. Similarly, 
ifp max (S, L, J-'swap) < a, then U s i s an a-approximate correlated equilibrium. 



3 Noise Tolerance of No-Regret Algorithms 

In this section we show that no-regret algorithms are tolerant to addition of 'some' noise, that is 
we still get good regret bounds with respect to the real losses if we run the no-regret algorithm on 
noisy losses (real losses plus low-magnitude noise). 

Let L G [0, l] Txk be a loss matrix. Define L = £±i (entrywise) and note that L G [~, |] TxA: . 
The following states that running A on L doesn't significantly increase the regret with respect to 
the real losses. 

LEMMA 1. For every algorithm A, every family T, and every loss matrix L G [0, l] Txfc , 

p(A(L),L,T)<3p(A(L),L,F)- 
In particular, for every L G [0, l] Txfc 



— ,18 log k — / 18 log k 

P(-Afixed(£), infixed) < -I/ — - — and p{A 5wap (L), L, J^p) < kd — - — . 

PROOF. Let 710, ...,7Tt G 11^ be any sequence of distributions and let /: {1, 2, . . . , k} —t 
{1,2,..., A;} be any function. Then 

p(7T , ...,TC T ,LJ)= A(7T , . . . , 7TT, L) - A(/07T , . . . , /07T T , L) 

= 3 (A(vro, . . . , 7T T , L) - A(/o7r , . . . , fon T , L)) 
= 3 (p(tt , . . . , tt t , L, /)) . 

The second equality follows from the definition of A and from linearity of expectation. The Lemma 
now follows by setting (ir , . . . , tt t ) = At{L), taking a maximum over / G T, and plugging in 
the guarantees of Theorem 4. □ 

In light of Lemma 1, for the rest of this section we will take L to be a loss matrix in r 2 ' 



This rescaling will only incur an additional factor of 3 in the regret bounds we prove. Let Z G 



3' 3J 

Txk 
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be a real valued noise matrix. Let L = L + Z (entrywise). In the next section we will consider the 
case where Z is an arbitrary matrix with bounded entries. Then we will prove a tighter bound for 
the case where Z consists of independent draws from a Laplace distribution. 

3.1 General Noise 

The next lemma states that when a no-regret algorithm is run on a noisy sequence of losses, it does 
not incur too much additional regret with respect to the real losses. 

Lemma 2 (Regret Bounds in the Presence of Bounded Noise). Let L G [|, |] Txfc be any loss 
matrix. Let Z = (zj.) G [—b,b] Txk be an arbitrary matrix with bounded entries, and let L = L + Z. 
Let A be an algorithm. Let J 7 be any family of functions. Then 

P (A(L), L, F) < p(A(L),L, F) + 2b. 

PROOF. Let (ir , . . . ,ir T ) be any sequence of distributions and let /: {1,2, ...,k} — > 
{1, 2, ...,&} be any function. Then: 

p(7T , . . . , 7T T , L, f) - p(7T , . . . , 7T T , L, f) 

= (A(7T , . . . , 7T T , L) - A(/OTT , . . . , /07T T , L)) - (A(tT , . . . , 7T T , L) - A(/o7T , . . . , /o7T T , L)). 

= (A(7T , . . . , 7T T , L) ~ A(7T , . . . , 7T T , L)) + (A(/OTT , . . . , /o7T T , L) - A(/o7T , . . . , fo7T T , L)) 
T k \ / T k \ 

fzZJ2 - %) + If E E(/ 07r *)^ - 3) ( b y definition of A ) 

v t=i j=i / V *=i i=i / 

V t=l j=l J \ t=l j=l J 



(by definition of z) (2) 

^ 6 ( ^ E E A + 6 ( h E £(/<>**)' ) * 14 1 < b) 



T K \ / ^ T K 



, t=l j=l ) \ t=l j=l 

= 26, 

where the final equality follows from the fact that n t , fon t are probability distributions. □ 

Corollary 1. Let L e [§, |] Txfc be any loss matrix and let Z G M, Txk be a random matrix such 
that F z [Z G [-b, b] Txk ] > 1 - (3 for some b G [0, §], and let L = L + Z. Then 



p(Axed(i"), L, J- fix e d ) >J^ + 2b 
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2. P; 



p(A swap (L) , L,J-< 



swap, 



3.2 Laplacian Noise 



Having handled the case of general noise, we will now prove a tighter bound on the additional 
regret in the case where the entries of Z are iid samples from a Laplace distribution. 

Lemma 3 (Regret Bounds for Laplace Noise). Let L e [|, |] Txfc be any loss matrix. Let Z = 



pTxk 



be a random matrix formed by taking each entry to be an independent sample from 
Lap (a), and let L = L + Z. Let A be an algorithm. Let T be any family of functions. Then for 
any r\ < a. 



P 

z 



< 2me-^ T/2Aa2 . 



p(A(L),L,F)-p(A(L),L,F)>r ] 

Proof. Let (n , . . . ,tt t ) be any sequence of distributions and let /: {1,2,..., A;} — > 
{1, 2, . . . , k} be any function. Recall by (2), 



T k 



T k 



p(vr , . . . , vr T , L, f) - p(vr , . . . , tt t , L, f) = I i *t z t ) + [? Y(f 01T t)j z t ) ■ ( 3 ) 

\ t=l j=l ) \ t=l j=l 

We wish to place a high probability bound on the quantities: 

^ T k 

t=l j=l 

Changing the order of summation, 



Y, 



ai,...,a T eA \t=l 




the equality follows by considering the following two ways of sampling elements z{. The first 
expression represents the expected value of z{ if t is chosen uniformly from {1,2,..., T} and 
then j is chosen according to n t . The second expression represents the expected value of z\ if 
(d , . . . , ax) are chosen independently from the product distribution m x n 2 x ■ ■ ■ x -k t and then a 4 is 
chosen uniformly from (a±, . . . , ot). These two sampling procedures induce the same distribution, 
and thus have the same expectation. Thus we can write: 



P nr > rj] < max P 

z ai ,...,a T eA z 



1 T 

t=l 



< P 



1 T 

t=i 



> r] 
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where the second inequality follows from the fact that the variables z\ are identically distributed. 
Applying Theorem 3, we have that for any rj < a, 



P % ,...,„ T > V] < e-" 2T/6<T2 



(4) 



Let (7r , . . . , n T ) = A(L). By Equation (3) we have 



p(A(L),LJ)-p(A(L),LJ)>ri 



< P 
z 



T k 



T 



t=l j=l 



T k 



t=l j=l 



where the last inequality follows from applying (4) to the sequences (tt , . . . , tt t ) and (/o7r , . . . , fo 
7Tt). The Lemma now follows by taking a union bound over T . □ 

Finally, we obtain a tighter counterpart of Corollary 1 when the noise is independent Laplacian 
noise. 



Corollary 2. Let L e [ 



1 2lTxfc 
3' 3J 



be any loss matrix and let Z e M Txfc be a random matrix 



formed by taking each entry to be an independent sample from Lap (a) for o < 6 wiKT/m 
L = L + Z. Then 



and let 



1. P; 



2. P; 



p(Ar me d{L), L, .Ffixed) > 



21ogfc + / 241og(4fc/£) 



p(A swap (L), L, J- swap ) > kJ^ + a- / 24fcIog(4fc//3) 



T 



< P- 



4 Private Equilibrium Computation 



Having demonstrated the noise tolerance of no-regret algorithms, we now argue that for appro- 
priately chosen noise, the output of the algorithm constitutes a jointly-differentially private mech- 
anism, in the sense of Definition 5. We prove two results of this type. First, in Section 4.1 we 
consider games with 'few' actions per player. While our algorithm for this case is conceptually 
more straightforward, it will not be sufficient in certain cases of interest. For example, in the 
routing games we described in the introduction, the set of actions available to a player consists 
of all routes between her starting point and her destination. Even if the graph (road network) is 
small, the number of feasible routes can be extremely large (exponential in the number of edges 
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(roads)). However, in such games, the set of types (utility functions) is small (i.e. the set of all 
source-destination pairs). Motivated by this observation, in Section 4.2 we consider games with 
large action spaces, but bounded type spaces. 

4. 1 Upper Bounds for Games with Few Actions 

To orient the reader at a high-level, our proof has two main steps. First, we construct a 'wrapper' 
NRLaplace- 4 which takes as input the parameters of the game, the reported tuple of utilities, 
and any no-regret algorithm A. This wrapper essentially runs the no-regret algorithm A in every 
period for each player on noisy losses, i.e. instead of reporting the true loss to A, it reports the loss 
plus appropriately chosen Laplacian noise. In Theorem 6 we argue that this constitutes a jointly 
differentially private mechanism in the sense of Definition 5. Then, in Theorem 7 and Corollary 
3, we argue that this wrapper converges to an approximate coarse correlated equilibrium when the 
input algorithm is *4fi xe d, and to an approximate correlated equilibrium when the input algorithm is 
A 

4.1.1 Noisy No-Regret Algorithms are Differentially Private 



NRLaplace- 4 (m, ...u n ) 
Params: e,5,j e (0,l],n,k,T e N 

LET: 711,1, . . . , n rhl each be the uniform distribution over {1,2,..., k}. 

Let . _ 7^8nfcTln(l/f) 

For: t = 1,2, ..!,T 

Let: l\ t = 1 — E 7r _ i t [ui(j, a_i)] for every player i, action j. 
Let: z\ t be an i.i.d. draw from Lap(cx) for every player i, action j. 
LET: l\ t = l{ t + z\ t for every player i, action j. 

Let: 7r M+ i = A(ir it t,Ti t t) for every player i. 
End For 

Output: (7^1, . . . , 7^) to player i, for every i. 



Theorem 6 (Privacy of NRLaplace- 4 ). For any A, the algorithm NRLaplace- 4 satisfies (e, 5) - 
joint differential privacy. 

PROOF. Fix any player i, any pair of utility functions for i, Ui, u'^ and a tuple of utility functions 
u_i for everyone else. To show differential privacy, we need to analyze the change in the distribu- 
tion of the joint output for all players other than i, (tT-^i, . . . , 7r_ ijr ) when the input is (v,i, U-i) as 
opposed to (n-, 
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It will be easier to analyze the privacy of a modified mechanism that outputs (/_»,!, . . . , l-i,r)- 
Observe that this output is sufficient to compute (tt-^i, . . . , ^-i,t) j ust by running A. Thus, if 
we can show the modified output satisfies differential privacy, then same must be true for the 
mechanism as written. 

For every player %' ^ i, action j 6 {1,2,..., k}, and t < T, we define the query Ql, t (- \ 
. . . on the utility functions as well as the output of the mechanism in rounds 

l,...,t-l. 

Query Q{, t (u h u_i \ L iA , . . . ,71 ijt _ x ) 

Using u^i, Ui and L^i, . . . , l-i t t-u compute l\, t . Observe that this can be done in the following 
steps: 

1. Using . . . ,l- ijt -u A, and u- if compute . . . , tt-^-i. 

2. Using n-i t i, ... , 7r_i >t _i, A and compute 71^1, . . . , n^t-i. 

3. Using 7r t _i = {n i)t -i, 7r_ M _i), A, and u i9 compute l\, t . 



Observe that the only step of the query computation that directly involves Ui is the second. 
Changing player i's utility function from m to u[ can (potentially) affect it^t-i, and can (poten- 
tially) alter it to an arbitrary state Tx^t-i- However, observe that 



Ql,(u h u_i I Z_ u , 



.,Li, t _i) = l- E [Mj,a_i')] 



E 



-(i,i'),t 



E aj, a-^i))] 



< 1 - E 

"■-(i'.O.t 



JE [v4'(j,ai,a-(ii } i)) + 7] 

Qi'A u 'i, u -i I • • • J-i,t-i) + 7 



where the inequality comes from the fact that m> is assumed to be 7-sensitive in the action of player 
i (Definition 1), and by linearity of expectation. A similar argument shows: 

Q\, t {ui,u-i I Lj,!, . . . ,T-i,t-i) > Ql.Jyu'^u-i I . . . ,T-i,t-i) - 7- 

Note two facts about these queries: (1) The answer to Q\, t is exactly l\, t , thus the noisy output 
to these queries (i.e. answer plus Lap(cx)) is indeed equal to the output of the (modified) algo- 
rithm NRLaplace- 4 . (2) The noisy losses l-n, . . . , l-a-i have already been computed when the 
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mechanism reaches round t, thus the mechanism fits the definition of adaptive composition. 

Thus, we have shown how to rephrase the output (U^i, . . . , U^t) as computing the answers to 
nkT (adaptively chosen) queries on (u\, . . . , u n ), each of which is 7-sensitive to the input Ui. Thus 



the Theorem follows from our choice of a = 7 £ 1 y^8nkT log(l/<5) and Theorems 1 and 2. □ 

4.1.2 Noisy No-Regret Algorithms Compute Approximate Equilibria 

Therefore we have shown how that the this 'wrapper' algorithm is jointly differentially private in 
the sense of Definition 5. We now proceed to show that using this algorithm with *4fi X ed will result 
in an approximate coarse correlated equilibrium (Theorem 7), and that using it with *4. SW a P will 
result in an approximate correlated equilibrium (Corollary 3). 

Theorem 7 (Computing CCE). Let A = *4.fi xe d- Fix the environment, i.e. the number of players 
n, the number of actions k, the sensitivity of the game 7, the degree of privacy desired, (e, 5), and 
the failure probability f3. One can then select the number of rounds the algorithm must run, T, 
satisfying: 

7£ -V8nmog (1 /*) < 6lo J ikmy (5) 

such that with probability at least I— [3, the algorithm NRLAPLACE" 4 *'^, returns an a-approximate 
CCE for: 10 

a = ( 7 e-Vwfclog(l/tf) log(l//3)) . (6) 

Before we proceed to the proof, some discussion is appropriate. It is already well known 
that no-regret alogrithms converge 'quickly' to approximate equilibria- recall Theorems 4 and 5. 
In the previous section, we showed that adding noise still leads to low regret (and therefore to 
approximate equilibrium). The tradeoff therefore is this. To get a more 'exact' equilibrium, the 
algorithm has to be run for more rounds. By the arguments in Theorem 6, this will result in a 
less private outcome. The current theorem makes precise the tradeoff between the two. Fixing the 
various parameters, (5) tells us the number of rounds T the algorithm must run for. Then, (6) tell 
us that fixing the desired privacy and failure probability, one can compute an a-approximate CCE 
for a = O^y/nk). 

This is a strongly positive result- in several large games of interest, e.g. anonymous matching 
games, 7 = 0(n^ 1 ). Therefore, for games of this sort a = 0{y/k/ Vn). If k is fixed, but n is 
large, therefore, a relatively exact equilibrium of the underlying game can be implemented, while 
still being jointly differentially private to the desired degree. 



LO 



Here O hides (lower order) poly (log n, logfc, logT, log(l/7), log(l/e), loglog(l//3), loglog(l/<5)) factors. 
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Proof of Theorem 7. By our choice of the parameter a, in the algorithm NRLaplace* 43 ", 
which is 



cr = 75 



-W8nkT\og(l/6), 



and by assumption of the theorem, (5), we have a < 1/6 log(4n/cT/ (3). Applying Theorem 2 we 
obtain: 



P 

z 



P\Ki,li ■ ■ ■ > TTi.T, -^i, ^fixed) > 



T 



< 



/3 



for any player z, where Lj is the loss matrix derived from the given utility functions U{ and the 
distributions {^i,t} i( z[ n ] t6 [yj- Now we can take a union bound over all players i, yielding: 



P 
z 



P 



max p(x 



8,1 



2 log fc y/24 log(4rafc//3) 



T 



(7 



^ 21ogfe v/241og(4nA;//3) 

Pmax(7T,L, J^ixed) > V — 7^ h O- 



T 



T 



T 



</3, 



By Theorem 5, therefore, the empirical distribution of play is a \J 2Xo ^ k + - 'v /241o ^ 4nfc /ft) _ 
approximate coarse correlated equlibrium. 

To finish, substitute o = ^e' -^y/ZnkT log(l/5) into the expression above. Therefore, with 
probability at least 1 — ft, no player has regret larger than 



1 2 log k -fJl92nk log(l/5) log(4nfc//3) 
a=W — + 

Since T is a parameter of the algorithm, we can choose T to minimize a. Since a is monotonically 
decreasing in T, we would like to choose T as large as possible. However, our argument requires 
(5), which (roughly) requires \/T < l/^yy/nk, where we have suppressed dependence on some of 
the parameters. By choosing T so that \/T ~ l/^\/nk we can make the first term of the error 
~ ^yy/nk, which would make it be of a similar order to the second term. It is easy to verify that we 
can choose T is such a way that T satisfies the assumption and the resulting value of a satisfies the 
conclusion of the theorem. □ 

By considering A swap instead of *4fi xe d, we easily get similar results for approximate correlated 
equilibrium rather than coarse correlated equilibrium. 



COROLLARY 3 (Computing CE). Let A = *4. SW ap- Fix the environment, i.e. the number of play - 
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ers n, the number of actions k, the sensitivity of the game 7, and the degree of privacy desired, 
(e, 5). One can then select the number of rounds the algorithm must run T, and two numbers a, (3 
satisfying: 

^-WSnkTl 0e{ l/ S )< 6lo J ikTm , (7) 

such that probability at least 1 — /3, the algorithm NRLAPLACE" 45 ™ 13 , returns an a-approximate 
correlated equilibrium for: 11 



a = 



7fc 3 /y™log(l/5)log(l//3) 



PROOF. Following the same steps as the Proof of Theorem 7, but noting that we are using regret 
with respect to rather than J-fi xe d, we find that NRLaplace- 45 ™ 311 will return, with probability 
at least 1 — j3, an a-approximate correlated equlibrium where 



l2\ogk AA;a/384™ log(l/5) log(4fcra//3) 

As in Theorem 7, we will choose T ~ l/^yfnk to complete the proof. □ 
4.2 Upper bounds for Games with Bounded Type Spaces 

Recall that in the previous section, we showed that a private equilibrium can be computed with 
a 0(Vk/y/n) approximate equilibrium. While these results are positive for some settings (e.g. 
anonymous matching games for large populations), they have no bite in settings where the number 
of actions is as large (or larger) than the number of players. The problem is roughly this- with 
large numbers of actions, the no-regret algorithm will have to be run 'many' times. This would 
require that we either sacrifice privacy, or introduce even more noise to ensure privacy, which in 
turn would give make the computed equilibrium a worse approximation. 

4.2.1 The Median Mechanism 

To keep notation straight, we will use u = (ui, . . . , Un) to denote the utility functions specified 
by each of the n players, and v E U to denote a utility function considered within the mechanism. 
Let U = \U\, the size of the set of possible utility functions for any player. In order to specify the 
mechanism it will be easier to define the following family of queries first. Let i be any player, j 
any action, t any round of the algorithm, and v any utility function. The queries will be specified 



"Again O hides lower order poly(log7V, log if, logT, log(l/A), log(l/e), loglog(l//3), loglog(l/<$)) factors. 
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by these parameters and a sequence A 1; . . . , A t _i where A t / e t nxi;xU for every 1 < t' < t — 1. 
Think of A^, as being the loss experienced by player i, if she decides to play action j, and her 
utility is v , given the state of the mechanism in round t' . 

Q?i,t,v{ u u • • • ,mtv | Ai, . . ., A t _i) 

Using Mi, . . . , un I Ai, . . . , A t _i, compute l\ t v = 1 — E 7T _ i t [ui(j, cm)]. This computation can 
be done in the following steps: 

1. For every i' ^ i, use K 3 V jl?v , . . . , A^ )t _ ljU . 7 , A and u*/ to compute n^^, ... ,71^^. 

2. Using 7r_ iit _i, compute l{ )t>v . 

Observe that Q\ t v is 7-sensitive for every player i, step t, action j, and utility function v. To 
see why, consider what happens when a specific player i' switches her input from to u' v . In that 
case that i = %', this has no effect on the query answer, because player i's utility is never used in 
computing Q\a v . In the case that i' ^ i then the utility function of player i' can (potentially) affect 
the computation of iti^t-i, an d can (potentially) change it to an arbitrary state T^ t-i- But then 7- 
sensitivity follows from the 7-sensitivity of Ui, the definition of P itv , and linearity of expectation. 
Notice that does not, however, affect the state of any other players, who will use the losses 
Ai, . . . , A t _x to generate their states, not the actual states of the other players. 

There are TnkU such queries. If we were to answer these queries with the Laplace mechanism, 
as in the algorithm NRLAPLACE, then we would introduce even more noise to ensure privacy 
of all the queries. However, in the case where U is small, we are able to use more privacy- 
efficient mechanisms, that can compute differentially private answers to the queries with much 
less noise than would be introduced by the Laplace mechanism. One such mechanism is the so- 
called Median Mechanism of Roth and Roughgarden [RR10], paired with the privacy analysis of 
Hardt and Rothblum [HR10]. 12 

Theorem 8 (Median Mechanism For General Queries [RR10, HR10]). Consider the following It- 
round experiment between a mechanism M. m, who holds a tuple 6 W, and a adaptive 
querier B. For every round r — 1,2, ... ,R: 

1. B(Qi, a\, ... , Qr-i, dr-i) = Qr, where Q r is a ^-sensitive query. 

2. a r i- R Mm(ui, ...,u n ; Q r ). 

12 Originally, the median mechanism of [RR10] was only defined and analyzed for the case of linear queries. A 
'folk' result, first observed by Hardt and Rothblum [Har] is that the Median Mechanism (when instantiated with a net 
of all possible size n datasets) can be applied to arbitrary 7-sensitive queries, which immediately yields Theorem 8 
when paired with the privacy analysis of [HR10]. The simple proof can be found in [DR13]. 
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For every e, 8, 7, /3 G (0, 1], iV, R, U G N, f/zere is a mechanism Mm such that for every E 

1. The transcript (Qi, ai, ■ ■ ■ , Qr, or) satisfies (e, 5) -differential privacy. 

2. With probability 1 — (3 (over the randomizations of Mm), \o, r — Qr(u\, . . . ,un)\ < ctM M 
for every r = 1,2, . . . , R and for 

a MM = 16e- 1 7v /iVlogf/log(2i?//3) log(4/5). 
4.2.2 Noisy No-Regret via the Median Mechanism 

Our mechanism uses two steps. At a high level, there is an inner mechanism, 
NRMedian-Shared, that will use the Median Mechanism to answer each query 
Qitv (' I ^l, • • • , A t _iJ , and will output a set of noisy losses Ai, . . . , Ay. The properties of the 
Median Mechanism will guarantee that these losses satisfy (e, 5) -differential privacy (in the stan- 
dard sense of Definition 4). 

There is also an outer mechanism that takes these losses and, for each player, uses the losses 
corresponding to her utility function to run a no-regret algorithm. This is NRMedian which takes 
the sequence Ai, . . . , At and using the utility function Ui will compute the equilibrium strategy 
for player i. Since each player's output can be determined only from her own utility function and 
a set of losses that is (e, 5) -differentially private with respect to every utility function, the entire 
mechanism will satisfy (e, <5)-joint differential privacy. 

NRMedian-Shared- 4 ^!, ...%) 

Params: e,5,j G (0,1], n, k,T e N 
For: t = 1,2, ...,T 

Let: l it,v = Mm (ui,...,u N -,Q{ !tiV (- I Ai, . . . , A t _i)) for every v. 

Let: A j (i, t, v) = for every i, j, v. 
End For 

Output: (Ai, . . . , A T ). 



Theorem 9 (Privacy of NRMedian). The algorithm NRMedian satisfies (e, 5)-joint differen- 
tial privacy. 

Proof. Observe that NRMedian can be written as h(u) = (/i(gr(u)), . . . , f N (g(u))) where f 
depends only on Ui for every player i. (Here, g is NRMedian- Shared and fi is the i-th iteration 
of the main loop in NRMedian). The privacy of the Median Mechanism (Theorem 8) directly 
implies that g is (e, 5) -differentially private (in the standard sense). 
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NRMedian- 4 (ui, . . . u N ) 

Params: e, 5, A e (0, l],n,k,T e N 

Let: (Ai, . . . , A T ) = NRMedian- Shared- 4 ^, . . . , u N ). 

For: i = l,...,N 

Let: 7^1 be the uniform distribution over {1,2,..., A;}. 
For: t = 1,...,T 

Let: 7r M = A (n^t-i, K,t-i,u,i) 
End For 

Output to player i: (ir it i, . . . , 7r i)T ). 
End For 



Consider a player i and two profiles u, u' that differ only in the input of player i, and consider 
the output (f_i(g(u))). Let S C Range(/_i) and let i?(u) = {o e Range(g) | f~\o) e S}. 
Notice that / is deterministic, so R is well-defined. Also notice that R depends only on S and u_. ; 
(in particular, not on Ui). Then we have 

P [h-\u) eS]= P [g(u) e R(u) = R(u')} 

h(u) g(u) 

< e e P [g(u) G R(u) = R(u')] + 8 

9(u') 

< e £ P [/i"Xu') 

/i(u) 

where the first inequality follows from the (standard) (e, 5) -differential privacy of g. Thus, NR- 
Median satisfies (e, 5)-joint differential privacy. □ 

4.2.3 Computing Approximate Equilibria 

Theorem 10 (Computing CCE). Let A be -4.fi xe d- Fix the environment, i.e the number of players 
n, the number of actions k, number of possible utility functions U, sensitivity of the game 7 and 
desired privacy (e, S). Suppose (3 and T are such that: 



We-^^n log U log(2nkTU/(3) log(4/5) < | (8) 

Then with probability at least 1 — /3 the algorithm NRMedian " 4fixed returns an a-approximate CCE 
for: 13 

n \y/N log 3/2 U \og(k/ (3) log(l/5)' 
a = U 



13 Here, O hides lower order poly(logn,loglogfc,logT,loglogJ71og(l/7),log(l/e),loglog(l//3),loglog(l/(5)) 
terms. 
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Again, considering 'low sensitivity' games where 7 is 0(l/n), the theorem says that fix- 
ing the desired degree of privacy, we can compute an a-approximate equilibrium for a = 
6 ^ (log ^i logfc ^ . The tradeoff to the old results is in dependence on the number of actions. The re- 
sults in the previous section had a \/k dependence on the number of actions k. This would have no 
bite if k grew even linearly in n. We show that positive results still exist if the number of possible 
private types is is bounded - the dependence on the number of actions and the number of types is 
now logarithmic. However this comes with two costs. First, we can only consider situations where 
the number of types any player could have is bounded, and grows sub-exponentially in n. Second, 
we lose computational tractability- the running time of the median mechanism is exponential in 
the number of players in the game. 

PROOF. By the accuracy guarantees of the Median Mechanism: 



P 
Mm 



3i,t,j,vs.t \lL v -lL v \ > A 



Mm 



where 



a Mu = 167£T V" log U \og(2nkTU//3) log(4/5) 
By (8), ol Mm < 1/6. Therefore, 



P 

Mm 



3i,j,t,v s.t. \li t ^ v h,t,v\ > 
Applying Theorem 1 and substituting Am m > we obtain: 



P 

z 



2 log k 

3i S.t. p(7T ii i, . . . , 7T i)T , L, J*fixed) > \/ j, h ^M M 



Now we can choose y/T = (^\/n) 1 to conclude the proof. 



□ 



Corollary 4 (Computing CE). Let A be ^. S wap- Fix the environment, i.e the number of players 
n, the number of actions k, number of possible utility functions U, sensitivity of the game 7, the 
desired privacy (e, 5), and the failure probability (3. Suppose T is such that: 



lGe-^^/n log U \og{2nkTU / (3) log(4/5) < 1 



(9) 



Then with probability at least 1 — /3 the algorithm NRMediaN" 45 ™^ returns an a-approximate CCE 
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for: 



.14 



a = 



log 3/2 tnog(£;//3)log(l/<5)' 



PROOF. By the accuracy guarantees of the Median Mechanism: 



P 

Mm 



3i,t,j,v s.t. \ll >t , v - lltj > A 



Mm 



where 



a MM = 16 7 e- Vri log U log(2nkTU / f3) log(4/<5) 
By (9), olmm ^ 1/6- Therefore, 



P 
Mm 



3i,j,t,us.t. - l 3 i>t>v \ > | 
Applying Theorem 1 and substituting a>j M , we obtain: 



</3 



P 
z 



3i S.t. p(7T ii i, . . . , 7Ti )T , L, Tfixed) > \/ + 2 «Mm 



2 log A; 



Now we can choose yT = k^y/n) 1 to conclude the proof. 



□ 



4.3 A Lower Bound 

In the case where 7 = 0(1/ 11) and k = 0(1), both of our algorithms from the previous Section 
compute a differentially private, a-approximate equilibrium for a ~ 1/ \/n (ignoring all other 
parameters). It is natural to ask whether or not we can achieve significantly smaller values of a 
using some other algorithm. In this section we prove a lower bound showing that this is not the 
case. Specifically, we show that there is no algorithm that privately computes an ^-approximate 
equilibrium of an arbitrary n-player 2-action game, for a 1 / \Jn log n. In other words, there 
cannot exist an algorithm that privately computes a 'signficantly' more exact equilibrium. 

Our proof is by a reduction to the problem of differentially private subset-sum query release, 
for which strong information theoretic lower bounds are known [DN03, DY08]. The problem is 
as follows: Consider a database D 6 {0, 1}™, which we denote (di, . . . , d n ). A subset-sum query 
q C [n] is defined by a subset of the n database entries and asks "What fraction of the entries in 
D are contained in q and are set to 1?" Formally, we define the query q as q(D) = ^ Yli^qdi- 



14 Here O hides iower order poly(logn,loglogfc,logT,loglogC/log(l/7),log(l/e),loglog(l//3),loglog(l/(5)) 
terms. 
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Given a set of subset-sum queries Q — {q 1} . . . , q m }, we say that an algorithm M.(D) releases Q 
to accuracy a if M(D) = (ai, . . . , a m ) such that \aj = qj(D)\ < a for every j £ [m\. 

Dinur and Nissim [DN03], showed that any differentially private algorithm that releases suffi- 
ciently many subset-sum queries must add a significant amount of noise. A quantitative improve- 
ment of their result is given by Dwork and Yekhanin [DY08]. They constructed a family Q DY of 
size m = 0(n) such that there is no differentially private algorithm that releases Q DY to accuracy 
o(l /y/n). Thus, a natural approach to proving a lower bound is to show that an algorithm for 
computing approximate equilibrium in arbitrary games could also be used to release arbitrary sets 
of subset-sum queries accurately. The following theorem shows that a differentially private mech- 
anism to compute approxmiate equilibrium implies a differentially private algorithm to compute 
subset-sums. 

Theorem 11. For any a > 0, if there is an (e,5)-jointly differentially private mechanism Ai 
that computes an a-approximate coarse correlated equilibria in {n + m log n) -player, 2-action, 
1/n-sensitive games, then there is an (e, 5) -differentially private mechanism Ai' that releases 36a- 
approximate answers to any m subset-sum queries on a database of size n. 

Applying the results of Dwork and Yekhanin [DY08], a lower bound on equilibrium computa- 
tion follows easily. 

COROLLARY 5. Any (e = 0(1), 5 = o(l)) -differentially private mechanism Ai that computes 
an a-approximate coarse correlated equilibria in n-player 2-action games with 0{l/n)- sensitive 
utility functions must satisfy a = fl( , \ ). 

Here, we provide a sketch of the proof of Theorem 11. Let D G {0,l} n bean n-bit database 
and Q = {q l7 . . . , q m } be a set of m subset- sum queries. For the sketch, assume that we have an 
algorithm that computes exact equilibria. We will split the (n + m) players into n "data players" 
and m "query players." Roughly speaking, the data players will have utility functions that force 
them to play "0" or "1", so that their actions actually represent the database D. Each of the query 
players will represent a subset-sum query q, and we will try to set up their utility function in such 
a way that it forces them to take an action that corresponds to an approximate answer to q(D). In 
order to do this, first assume there are n + 1 possible actions, denoted {0,-,-,...,l}. We can 
set up the utility function so that for each action a, he receives a payoff that is maximized when 
an a fraction of the data players in q are playing 1. That is, when playing action a, his payoff is 
maximized when q(D) = a. Conversely, he will play the action a that is closest to the true answer 
q(D). Thus, we can read off the answer to q from his equilibrium action. Using each of the m 
query players to answer a different query, we can compute answers to m queries. Finally, notice 
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that joint differential privacy says that all of the actions of the query players will satisfy (standard) 
differential privacy with respect to the inputs of the data players, thus the answers we read off will 
be differentially private (in the standard sense) with respect to the database. 

This sketch does not address two important issues. The first is that we do not assume that the 
algorithm computes an exact equilibrium, only that it computes an approximate equilibrium. This 
relaxation means that the data players do not have to play the correct bit with probability 1, and the 
query players do not have to choose the answer that exactly maximizes their utility. In the proof 
we show that the error in the answers we read off is only a small factor larger than the error in the 
equilibrium computed. 

The second is that we do not want to assume that the (query) players have n + 1 available 
actions. Instead, we use logn players per query, and use each to compute roughly one bit of the 
answer, rather than the whole answer. However, if the query players' utility actually depends on 
a specific bit of the answer, then a single data player changing his action might result in a large 
change in utility. In the proof, we show how to compute bits of the answer using sensitive 
utility functions. 

4.4 Incentive Properties 

One of the things touched upon in our introduction was the incentive properties of our proposed 
mechanism. It is well understood that differentially private mechanisms are also approximately 
strategy proof (This point was initially made in McSherry and Talwar [MT07]). This will give 
us the desired incentive properties in our setting as well. The basic idea is as follows: Fix some 
player i considering changing his report. Joint differential privacy implies that fixing the reports of 
the other players, for any report of player i, the distribution over actions suggested to players —i 
cannot change 'much'. Therefore player z's gain from misreporting must also be small. Formally, 
we have the following theorem: 

Theorem 12. Consider a (e, 5)-jointly differentially private mechanism Ai which computes a a- 
correlated equilibrium of the full information game induced by players' reports. Then: 

1. If all players must follow their recommended actions, then it is a (e e — 1) + 6 -approximate 
dominant strategy for each player to report their type truthfully. 

2. It is a (e e — 1) + 5 + a-approximate Nash Equilibrium for players to each play the following 
strategy- "truthfully report your type to the mechanism, then follow the suggested action ". 

Part 1 follows easily from the definition of joint differential privacy and the fact that payoffs 
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are bounded between and 1. Part 2 follows since the mechanism suggests an a-approximate 
correlated equilibrium to the players. 

It is easy to select e, 5 and a so that the incentive properties are also 'good' for large games. In 
particular recall that a is 0{y/\og{l/5)/ey/n) (Corollary 3). Selecting e.g. e of O^- 1 / 4 ), and 5 
of 0(l/n), we have a is 0(n _1//4 ). Therefore for large n, the loss from privacy and approximation 
of equilibrium computed by this mechanism will asymptote to 0. Further it will be an almost exact 
equilibrium for all players to truthfully report their type and then follow the suggested action- the 
approximation is e + a + 5 = 0(n -1 / 4 ). 
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A Proofs 



A.l Proofs from Section 3 



Proof of Corollary 1. We will prove only item 1, the proof for 2 is analogous. First, by 
the assumption of the theorem, we will have L G [0, l] Txfe except with probability at most (3. 
Therefore, by Theorem 4, 



P 

z 



2 log A; 
T 
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Further, by Lemma 2, we know that L £ [0, l] Txfc implies 



p(Axed(£), £, -T) < p(A rixed (L), L, F) + 2b. 



Combining, we have the desired result, i.e. 



P 

z 



2 log k 

p(A f \ xed (L), L, J'fixed) > \j — V 2b 



□ 



Proof of Corollary 2. First, we demonstrate that L e [0, l] Txk except with probability at 
most (3, which will be necessary to apply the regret bounds of Theorem 4. Specifically: 



P 

z 



3zl s.t. \zt\> 



1 



< TfcP 

z 



z > 



1 



< 2Tke- 1/6a < /3/2, 



(10) 



where the first inequality follows from the union bound, the second from the definition of Laplacian 
r.v.'s and the last inequality follows from the assumption that a < 1/6 \og(ATk/ f3). 

The Theorem now follows by conditoning on the event L G [0, l] Txfc and combining the regret 
bounds of Theorem 4 with the guarantees of Lemma 3. For parsimony, we will only demonstate 
the first inequality, the second is analogous. Recall again by Theorem 4, we have that whenever 
Te [0,l] Txfc : 



P[Af K ed(L), L, .Ffixed) < 



2 log A; 



T 



Further, by Lemma 3, we know that: 



P 
z 



P(Af lxe d(L) , L, Jlixed) — p(*4fi X ed(£), L, J-fixed) > 1] 



< 2\^ lxed \e^ T ^ 2 
= 2ke-^ T ' 2 ^. 



Substituting i] = a 



24 log(4fc//3) 



we get: 



P 
z 



p(Af K ed(L), L, J'fixed) - p(Afi xe d(L), L, J'fixed) > V < (3/2 



The result follows by combining (10) and (11). 



(ID 
□ 
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A. 2 Proofs from Section 4 



A. 3 Proof of Theorem 1 1 

Given a database D G {0, l} n , D = (d\, . . . , d n ) and m queries Q = {g l5 . . . , q m }, we will 
construct the following (N = n + mlogn)-player 2-action game. We denote the set of actions 
for each player by A = {0, 1}. We also use {(j, h)}j e , m i he n ogn \ to denote the mlogn players 
{n + 1, . . . ,n + mlogn}. For intuition, think of player (j, ft,) as computing the h-th bit of qj(D). 
Each player i G [n] has the utility function 

I 1 if di = di 

Ui{a) = < 

I otherwise 

That is, player i receives utility 1 if they play the action matching the i-th entry in D, and utility 
otherwise. Clearly, these are 0-sensitive utility functions. 

The specification of the utility functions for the query players (j, h) is somewhat more compli- 
cated. First, we define the functions fh,gh- [0,1] — > [0, 1] as 

f h (x) = 1 - min \x- (2- {h+1) + r2- ( ^ 1} ) I 

re{0,...,2' l - 1 -l} 

g h (x) = l- min \x - {2~ h + 2- {h+1) + r2- {h - 1) )\ 

re{0,...,2' l - 1 -l} 

Each player (j, h) will have the utility function 

U {j,h)( a -U,h)> °) = facial, On)) 

u U,h)( a -U,h), 1) = 9h(qj(ai, «n)) 

Since q(ai, . . . , a n ) is defined to be 1/n-sensitive in the actions a\,...,a n , and fh, gh are 1- 
Lipschitz in x, is also 1/n-sensitive. 

Also notice that since Q is part of the definition of the game, we can simply define the set of 
feasible utility functions to be all those we have given to the players. For the data players we only 
used 2 distinct utility functions, and each of the mlogn query players may have a distinct utility 
function. Thus we only need the set U to be a particular set of utility functions of size m log n + 2 
in order to implement the reduction. 

Now we can analyze the structure of ct-approximate equilibrium in this game, and show how, 
given any equilibrium set of strategies for the query players, we can compute a set of 0(a)- 



36 



approximate answers to the set of queries Q. 

We start by claiming that in any a-approximate CCE, every data player players the action d{ in 
most rounds. Specifically, 

Claim 1 . Let ir be any distribution over A N that constitutes an a-approximate CCE of the game 
described above. Then for every data player i, 

P di] < a. 

TT 

Proof. 

(Definition of a-approximate CCE) 
(Definition of Ui) □ 

The next claim asserts that if we view the actions of the data players, a 1( . . . , a n , as a database, 
then q(ai, . . . , a n ) is close to q(d\, . . . , d n ) on average. 

Claim 2. Let tt be any distribution over A N that constitutes an a-approximate CCE of the game 
described above. Let q C [n] be any subset-sum query. Then 

E[\q(d 1 ,...,d n ) - q(a t , . . . , a n )\) < a. 

7T 

Proof. 

E[\q(d 1 ,...,d n ) - q(a x , . . .,a„)|] = E 

7T 7T 

< ~y2E[\d i -a i \] = -y2F[a i ^d i ] 

< - Yl a - a (Claim l,gC[n]) □ 

i£q 

We now prove a useful lemma that relates the expected utility of an action (under any distribu- 
tion) to the expected difference between qj{a%, . . . , a n ) and qj(D). 



P oi ^di\ = 1 -E u^a-i) 

7T 7T 

< 1 - ^E [^(dj^-j)] - a; J 
= 1 — (1 — a) — a 



-y~){di- a d 

n L — 4 

i&q 
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CLAIM 3. Let // be any distribution over A . Then for any query player (j, h) 



E [uo»(°)°-(j»)] - fh(<lj( D )) 
E [u(i,fc)(l,0-(j,fc))] -9h(qj(D)) 



<E[|?,-(oi,... 



<&(£>)|], and 



<E[|^(ai,...,a n )-^(D)|]. 



Proof. We prove the first assertion, the proof of the second is identical. 

E[u m {0,a^)] -f h { qj {D)) 
E[f h (q j (a 1 ,...,a n ))-f h (q j (D))} 



< E [|q,-(ai, . . . ,a„) - 



(/h is 1-Lipschitz) 



□ 



The next claim, which establishes a lower bound on the expected utility player (j, h) will obtain 
for playing a fixed action, is an easy consequence of Claims 2 and 3. 

Claim 4. Let tt be any distribution over A N that constitutes an a-approximate CCE of the game 
described above. Then for every query player (j, h), 



E[u m (0,a^)]-f h (qj(D)) 
E [u {jjh) (l,a-i)] - g h {q j {D)) 



< a, and 

< a. 



Now we state a simple fact about the functions fh and g^. Informally, this asserts that we 
can find alternating intervals of width nearly 2~ h , that nearly partition [0, 1], in which fh{x) is 
significantly larger than gh(x) or vice versa. 

Observation 1. Let p < 2~ (/l+1) . If 

xe [j (r2- h + /3,(r + l)2- h -/3) 

re{0,l,...,2' l - 1 -l} 

then fh(x) > gh(x) + (5. We denote this region F h ^. Similarly, if 

xe |J ((r + l)2- fe + /?, (r + 2)2- fe - 0) 

re{Q,l,...,2 A - 1 -l} 



then ^(a;) > fh(x) + /3. We denote this region Gh,p 
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For example, when h = 3, F 3 ^ = [0, ± - /3] U [§ + /3, | - /?] U [| + /?, § - /?] U [| + /?, f - 
By combining this fact, with Claim 4, we can show that if qj(D) falls in the region F h a , then 
in an a-approximate CCE, player (j, h) must be playing action 'often'. 

Claim 5. Let n be any distribution over A N that constitutes an a-approximate CCE of the game 
described above. Let j G [m] and 2~ h > 10a. Then, if q^D) G F hiBa , P ff [a* = 0] > 2/3. 
Similarly, if q^D) G G Ma , then P w [a< = 1] > 2/3. 

PROOF. We prove the first assertion. The proof of the second is identical. If player (j, h) plays 
the fixed action 0, then, by Claim 4, 

E [u(,-,h)(0,a_(,-, h ))] > h(qj(D)) - a. 

Thus, if 7r is an a -approximate CCE, player (j, /i) must receive at least fh(qj(D)) — 2a under 7r. 
Assume towards a contradiction that P [a^) = 0] < 2/3. We can bound player (j, h)'s expected 
utility as follows: 

E [u (jth )(&)] 
= P [a {jA) = 0] E [M (j>) (0,a_ ( ^)) | a (j>) = 0] 
+ P [%» = l] E [M (ji/l) (l,a_( iift )) | au h ) = l] 

<F[a (j , h) = 0](f h (q j {D)) + E [^-(ax, . . . , a n ) - qj (D)\ | a m = 0] 
+ P[a 0Vl) = l] (^(D))+ E [|^(oi,...,a n )-^(D)||o ( ,- h) = l] 

\ a^RTr 

= f h ( qj (D)) + E [\ qj ( ai , ...,a n )- qj (D)\) - P [a m = l] (/ fc fo(I>)) - g h {qj{D))) 

a<- R 7T 

< //>(&(£>)) + a - 9aF[a m = l] (13) 
<f h (q j (D))-2a (14) 

Line (12) follows from the Claim 3 (applied to the distributions n | ay m = and ir \ au t h) = 1). 
Line (13) follows from Claim 2 (applied to the expectation in the second term) and the fact that 
qj(D) G Fh^a (applied to the difference in the final term). Line (14) follows from the assumption 
that P [a(j,h) = 0] < 2/3. Thus we have established a contradiction to the fact that n is an a- 
approximate CCE. □ 

Given the previous claim, the rest of the proof is fairly straightforward. For each query j, 
we will start at h — 1 and consider two cases: If player (j, 1) plays and 1 with roughly equal 



(12) 
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probability, then we must have that qj(D) £ F 19q U Gi^q,. It is easy to see that this will confine 
qj(D) to an interval of width 18a, and we can stop. If player (J, 1) does play one action, say 0, a 
significant majority of the time, then we will know that qj (D) E F lj9a , which is an interval of width 
1/2 — 9a. However, now we can consider h — 2 and repeat the case analysis: Either (j, 2) does not 
significantly favor one action, in which case we know that qj(D) ^ F 2> g a U G 2 $ a , which confines 
qj(D) to the union of two intervals, each of width 18a. However, only one of these intervals will 
be contained in which we know contains qj(D). Thus, if we are in this case, we have learned 
qj(D) to within 18a and can stop. Otherwise, if player (j, 2) plays, say, a significant majority of 
the time, then we know that qj(D) G Fi t g a fl F 2:9a , which is an interval of width 1/4 — 9a. It is 
not too difficult to see that we can repeat this process as long as 2~ h > 18a, and we will terminate 
with an interval of width at most 36a that contains qj(D). 

Remark 13. We remark that we used 0(n) linear queries in proving our lower bound, for 
which a lower bound of VL{1/ ^/n) is known for (e, 5) -differentially private algorithms. Thus, our 
Vt{l/\/n log n ) lower bound also applies to games with linear utility functions. However, stronger 
lower bounds of Vt{l) are known for answering 0(n) low sensitivity nonlinear queries on a binary 
valued database [Del2] while preserving (e, 0) -differential privacy. We could equally well use the 
queries from the lower bound argument of [Del2] in our construction, to show that no (e, 0)-jointly 
differentially private algorithm can compute an a-approximate CCE to an n-player, 2-action, sen- 
sitivity 1/n game for any a < c, where c is some fixed universal constant. This proves a strong 
separation between (e, 5)-private equilibrium computation for 5 > 0, and (e, 0)-private equilibrium 
computation. In particular, with (e, 0)-privacy, it is not possible to compute an approximate equi- 
librium where the approximation factor tends to with the number of players, and therefore not 
possible to get the "strategyproofness in the large" results that we are able to obtain when 5 > 0. 
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